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ABSTRACT 


This  interim  report  summarizes  the  research  performed  under  Contract 
No.  F30602-78-C-0057  by  the  Polytechnic  Institute  of  New  York  for  Rome  Air 
Development  Center  from  January  1,  1978  to  January  31,  1979. 

Recursive  function  theory  was  applied  to  the  problem  of  program  com- 
plexity. This  study  was  completed  and  a technical  report  was  Issued.  The 
present  report  contains  the  abstract  of  the  technical  reports. 

The  Inquiry  Into  the  number  of  tests  necessary  to  verify  a computer 
program  was  undertaken.  One  phase  of  this  study  was  completed,  and  a tech- 
nical report  was  Issued.  The  present  report  contains  the  abstract  of  the 
technical  report. 

A study  was  undertaken  of  software  test  models  and  of  the  implementation 
of  associated  test  drivers.  The  present  report  describes  this  work  as  well 
as  the  test  drivers  obtained  so  far.  A new  approach  to  statistical  aspects 
of  program  testing  has  been  taken.  The  assumptions  and  the  resulting  model 
are  described  in  the  present  report. 

A new  measure  of  complexity  based  upon  information  theory  is  introduced. 
This  measure  assumes  that  a language  feature  used  Infrequently  is  more 
likely  to  be  used  Incorrectly  than  a language  feature  used  frequently. 

The  measure  has  the  advantage  of  being  sensitive  to  the  different  levels 
of  nestings  In  either  IF'S,  DO'S,  or  procedures. 

A number  of  different  schemes  are  suggested  for  the  calculation  of  the 
measure.  A method  for  automatic  calculation  of  the  measure  at  an  Installa- 
tion Is  also  discussed. 

The  relation  between  program  complexity  and  the  program's  Information 
content  was  also  investigated.  The  results  obtained  so  far  are  described 
In  this  report. 

Two  models  for  the  management  of  software  were  Investigated.  The  first 
one  models  the  productivity  (measured  In  Instructions  per  tenths),  as  well 
as  the  man-months  required.  The  second  model  investigates  different  communi- 
cation schemes  that  can  be  evolved  when  a problem  Is  partitioned  into 
several  subproblems. 

The  concluding  section  of  the  report  describes  the  planned  work  in  the 
next  period  and  lists  professional  activities  of  the  personnel  during  the 
present  reporting  period. 
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1.0  INTRODUCTION 

Thl*  Interim  report  summarizes  the  research  performed  under  Contract  Ho. 
F30602-78-C-0057  by  the  Polytechnic  Institute  of  New  Tork  for  Rome  Air  Devel- 
opment Center  from  January  1,  1978  to  January  31,  1979.  The  major  topics 
Investigated  and  summaries  of  the  progress  to  date  are  described  In  Chapter 
2.0;  the  references  are  listed  in  Chapter  3.  In  Chapter  4 the  participating 
personnel  are  listed,  as  well  as  their  activities  during  the  reporting 
period.  In  Chapter  5 an  outline  Is  given  of  the  planned  direction  of  the 
research  in  the  Immediate  future.  Chapter  6 reports  on  the  activities  of 
the  staff  during  1978. 

2.0  SUMMARY  OF  PROGRESS 

This  chapter  summarises  the  work  performed.  Upon  the  completion  of  each 
task  a report  on  the  task's  results  Is  Issued.  Several  technical  reports 
which  document  either  a completed  task  or  a phase  In  a continuing  effort  are 
in  various  stages  of  preparation. 

2.1  STUDY  OP  RECURSIVE  FUNCTION  THEORY  AMD  ITS  APPLICATION  TO  PROGRAM 

COMPLEXITY  — by  A.E.  Laaamel 

This  study  was  completed  and  a technical  report  submitted  in  May  of  1978 
(revised  version  in  October,  1978).  The  Abstract  Is  presented  below. 

Abstract 

It  is  desirable  to  have  a measure  of  complexity  for  a computer  program 
because  more  complex  programs  can  be  expected  to  cost  more  to  produce,  to 
require  more  time  to  test  and  debug,  and  to  have  more  residual  bugs  even 
after  testing  and  debugging.  Presumably  a program  need  be  complex  only  to 
the  extent  that  the  problem  It  Is  solving  Is  complex  so  that  a way  of  cal- 
culating problem  complexity  might  be  expected  to  lead  to  a method  of 
predicting  programming  and  debugging  costs.  The  purpose  of  this  report  la  to 
examine  the  extent  to  which  certain  well-defined  measures  of  complexity  In 
recursive  function  theory  might  fulfill  these  expectation.  The  conclusion 
reached  la  that  the  complexity  defined  by  recursive  function  theory  (or  an 
equivalent  computability  theory)  is  not  too  useful  as  a measure  of  programm- 
ing difficulty. 

Theories  of  computability.  In  the  1930's  and  1940's  mathematicians  and 
logicians  began  to  realize  that  certain  problems  which  were  very  precisely 
stated  were  not  just  difficult  but  Impossible  to  solve.  In  order  to  make 
such  a strong  assertion,  the  exact  computational  procedures  which  were  to 
be  allowed  had  to  be  explicitly  described.  Of  course.  If  someone  than 
discovered  a more  powerful  procedure  an  "unaolvable"  problem  might  well 
become  solvable.  However,  although  about  half  a dozen  different  models  of 
computation  were  developed,  they  all  lead  to  exactly  the  same  class  of 
solvable  problems.  This  class  Is  the  general  recursive  functions  In  the 
terminology  of  one  of  the  computation  models  due  to  Godel,  Kleene,  and  others. 


Degrees  of  computability  Given  that  certain  problems  can  be  solved  and 
others  cannot,  or  that  certain  functions  can  be  computed  effectively  while 

others  cannot,  can  one  then  refine  this  go/no-go  dichotomy  to  a scale  of  <j 

difficulties  of  computation  for  solvable  problems?  The  answer  turns  out  to 
be  yes,  especially  if  recursive  function  theory  is  taken  as  the  basic  model. 

In  particular,  the  primitive  recursive  functions  represent  a more  easily 
computed  subset  of  the  general  recursive  functions,  and  the  Grzegorzyk 
classes  partition  the  primitive  recursive  functions  according  to  a certain 
definition  of  "difficulty  of  computation". 

Relation  to  programming  difficulties  The  following  question  now  arises:  ! 

Granted  that  a connection  has  been  made  between  problem  complexity  and 

difficulty  of  computation,  can  this  connection  be  extended  to  the  difficulty  ■ 

of  programming  the  computation  which  is  to  solve  the  problem?  A case  is  j 

made  in  this  report  for  a "no"  answer  to  this  question. 

i 

The  report  reviews  some  theories  of  computational  complexity,  together 
with  some  estimates  of  their  importance  to  software  reliability.  An  attempt 
has  been  made  to  emphasize  the  practical  rather  than  the  theoretical. 

Examples  are  shown  of  very  complex  programs  (in  the  sense  of  looping,  time 
required,  etc.)  which  should  lead  to  few  programming  errors  because  the 
programming  language  allows  a direct  copying  of  the  algorithm  from  a 
standard  text  book. 


2.2  ON  THE  NUMBER  OF  TESTS  NECESSARY  TO  VERIFY  A COMPUTER  PROGRAM  — by 

G.S.  Pop kin  and  M.L.  Shooman 

This  study  was  completed  and  a report  issued  in  June  of  1978.  The 
abstract  is  presented  below. 

Abstract 

This  report  discusses  various  aspects  of  verifying  that  a computer  pro- 
gram correctly  carries  out  some  specified  functions.  If  the  program  was  de- 
signed with  the  aid  of  a flowchart,  the  flowchart  can  be  used  to  determine 
the  number  of  tests  necessary  to  verify  the  program.  If  the  program  was 
prepared  without  a flowchart,  then  either  a flowchart  or  a directed  graph 
must  be  prepared  from  the  program  to  determine  the  number  of  tests. 

One  way  to  verify  a computer  program  is  to  run  it  with  sample  input  data 
and  examine  the  results  of  correctness,  i.e.,  test  for  agreement  between  the 
program  output  and  the  results  that  would  be  produced  by  correct  execution 
of  the  specification  on  the  same  data.  This  requires  prior  determination  via 
hand  computation  or  other  independent  means  of  the  correct  outputs  for  the 
sample  inputs.  It  is  useful  for  us  to  first  classify  the  different  types  of 
tests  which  we  will  be  discussing.  A level  zero  test  is  defined  as  a test 
consisting  of  one  or  more  test  cases  which  together  cause  every  program 

statement  to  be  executed  at  least  once.  J 

Before  defining  a level  one  test,  it  is  necessary  to  define  a program 
segment,  which  we  shall  do  in  terms  of  flowchart  terminology.  In  general  our 
flowcharts  will  consist  of  one  start  and  one  stop  terminal  (ovals),  pro- 
cessing elements  (rectangles  or  parallelograms),  and  decision  elements  also 
called  deciders  (diamond  shape).  A segment  is  any  flow  sequence: 
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a)  from  the  START  terminal  to  the  first  decider, 

b)  from  the  exit  of  one  decider  to  the  entry  of  another, 

c)  from  the  last  decider  to  the  STOP  terminal, 
whether  or  not  any  of  these  flows  contain  processing  steps. 

A subroutine  Is  considered  part  of  a segment  if  the  segment  contains  a 
call  to  the  subroutine.  Segments  may  overlap,  l.e.,  a processing  step  may  be 
contained  in  more  than  one  segment.  If  a program  contains  more  than  one  TOP 
terminal,  then  a group  of  segments  Is  formed  by  the  flow  to  each  terminal 
from  the  respective  last  decider. 

In  a level  one  test,  all  flowchart  paths  are  force-traversed  at  least 
once.  In  a level  two  test  every  program  path  is  naturally  executed  at  least 
once.  Level  two  and/or  level  three  tests  have  formerly  been  called  "exhaus- 
tive tests,"  but  new  definitions  (given  In  the  following  section)  are  more 
precise  and  should  replace  the  earlier  usage.  There  are  some  particular 
types  of  problems,  especially  those  dealing  only  with  integer  variables,  that 
have  a sufficiently  limited  set  of  input  possibilities  to  make  a level  twp  or 
level  three  test  practical.  However,  in  general  level  two  and  three  tests 
are  intractable. 

In  this  report,  attention  will  be  directed  mainly  to  level  one  and 
related  types  of  tests.  That  is,  we  will  discuss  tests  which  are  orien- 
ted to  the  testing  of  program  segments.  Drawing  on  the  work  of  M.  Lipow 
(1),  it  will  be  shown  how  graphs,  matrices,  and  zero-one  integer  linear 
programming  (2)  can  be  used  to  determine  the  number  of  test  cases  needed 
to  perform  a level  one  test,  and  the  data  needed  to  comprise  those  tesJ- 
cases. 


The  approach  consists  of  finding  the  maximum  incomparable  set,  i.e., 
the  largest  set  of  program  segments  through  which  one  and  only  one  test 
case  should  pass.  The  size  of  this  set  gives  the  minimum  number  of  tests 
necessary  to  execute  each  segment  at  least  once,  while  the  elements  of 
this  set  give  the  paths  of  each  test. 

In  addition,  a discussion  is  given  of  methods  for  estimating  the  number 
of  paths  in  a loopless  flowchart. 


2-3  SOFTWARE  TEST  MODELS  AKD  THE  IMPLEMENTATION  OF  ASSOCIATED  TEST 
DRIVERS  — by  D.L.  Baggi  and  M.L.  Shooman 

This  study  Is  nearing  completion  and  a technical  report  Is  being  pre- 
pared. Parts  of  the  work  In  progress  will  be  presented  below. 


2.3.1  Abstract 

In  the  past  most  software  tests  were  constructed  by  heuristics  and 
drawing  upon  experience  with  similar  software.  Recently,  enough  preliminary 
work  has  Seen  done  to  propose  the  analytical  construction  of  test  cases.  i 

This  report  begins  by  defining  five  broad  classes  of  software  tests: 

Type  0,  Type  1,  Type  2,  Type  3 and  Type  4.  Types  0,1,  and  2 (l.e. , level  • 

0,1,  and  2)  were  defined  In  the  proceeding  section.  A Type  3 test  Is  f 

defined  as  an  exhaustive  Interaction  of  all  values  of  Input  and  stored 

variables  with  all  paths.  In  a Type  4 test,  the  sequence  of  Input  and 

stored  variables  must  also  be  considered.  Clearly  Type  3 and  4 tests  are 

unfeasible  and  only  tests  lying  between  Type  1 and  2,  utilizing  a sampling 

from  the  variable  space,  are  realistic. 

In  the  case  of  a Type  2 teat  one  must  still  be  able  to  enumerate  the 
paths  In  the  program  and  decide  on  a set  of  test  Inputs  which  will  traverse 
each  path  at  least  once.  This  is  far  from  a trivial  problem  in  the  case  of  a 
large  program.  This  report  presents  lower  and  upper  bounds  on  the  number  of 
paths  In  a program  as  a function  of  the  number  of  deciders.  These  bounds 
serve  as  a first  estimate  of  the  work  involved.  A decomposition  algor ithn  Is 
then  given  which  allows  a graph  to  be  cut  Into  many  smaller  subgraphs.  Fur- 
thermore, a second  algorithm  has  been  developed  which  can  be  programmed  to 
machine- Identify  all  paths  of  a program  under  test. 

An  Implementation  of  a complete  driver  of  Type  1.5,  that  is,  of  type 
higher  than  1,  Is  then  thoroughly  described,  together  with  the  algorithms  to 
define  paths  and  force  execution.  The  algorithm  has  been  Implemented  and  a 
driver  program  designed  to  force  testing  of  PL/1  programs;  however,  it  Is 
shown  that  these  techniques  could  be  extended  to  almost  any  programing 
language. 

An  example  of  a program  run  with  analytical  debugging  techniques  will 
also  be  considered  and  some  evaluation  of  the  usefulness  of  the  system  will 
eventually  be  given  In  the  light  of  the  accumulated  experience. 


2.3.2  Introduction 

At  the  present  state  of  the  programing  art,  there  exist  two  techniques 
for  removing  errors  from  a program  during  various  stages  of  development, 
program  proofs  and  program  testing.  Although  much  effort  has  been  expended 
on  program  proofs,  it  Is  not  clear  whether  this  method  will  become  a prac- 
tical and  widely  used  technique.  The  present  universally  used  technique  Is 
to  test  to  remove  bugs,  either  by  code  reading,  by  walkthroughs,  or  by  ma- 
chine testing. 
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In  order  to  Investigate  a strategy  for  testing  — be  It  manual,  semi- 
automatic or  automatic  — It  Is  necessary  to  provide  some  theoretical  back- 
ground, such  as  formal  definitions  and  analytic  models,  to  fully  define  the 
range  and  scope  of  the  test  project.  In  general,  it  is  indeed  unclear  what 
really  is  meant  by  error  models,  debugging  procedures,  and  other  such  terms. 
We  describe  here  a hierarchy  of  testing  models.  The  Importance  of  testing 
cannot  be  exaggerated,  because  only  a well-tested  program  can  be  assumed  to 
be  reasonably  error-free.  In  the  prevailing  lack  of  general  techniques  to 
prove  the  correctness  of  procedures. 

Much  of  the  testing  presently  done  is  ad  hoc.  and  heuristic  rather  than 
having  any  theoretical  background.  The  purpose  of  this  paper  is  to  present 
some  models  and  analytical  techniques  which  can  be  used  in  developing  soft- 
ware test  systems.  It  will  be  shown,  from  formal  definitions  of  testing 
types,  that  practical  driver  systems  for  automatic  testing  can  be  im- 
plemented. 

The  test  type  to  be  discussed  in  detail  is  a Type  1 test,  which  is  de- 
fined as  a test  model  in  which  each  program  path  is  force-traversed  once. 

The  definition  Involves  a discussion  on  how  program  branching  points  and 
loops  affect  the  number  of  paths.  The  process  culminates  in  an  algorithm  for 
identifying  all  program  paths. 

The  possibility  of  implementing  and  automating  such  a testing  model  is 
then  investigated.  It  is  shown  that  the  technique  is  feasible;  a system  of 
programs  has  been  implemented  to  force  execution  through  all  possible  paths 
of  a given  program  under  test.  This  requires  that  the  system  analytically 
determine  all  program  paths  from  the  code,  modify  the  input  code  and  drive 
several  runs  of  the  program.  Study  of  these  forced  runs  will  result  in  many 
program  errors  being  caught  without  having  to  calculate  and  Insert  particu- 
lar testing  data;  definitely,  a major  effort  if  done  by  hand  for  a complex 
program.  The  computer  output  for  each  run  contains  a unique  labeling  of  the 

path  traversed,  related  error  messages  and  normal  output^,  if  any,  and  the 
amount  of  time  elapsed  during  that  run.  The  system  has  already  proved  very 
valuable  in  program  debugging  on  a few  problems,  with  its  fully  automatic 
mode  of  operation  being  the  significant  asset. 

In  the  following  pages  we  will  first  describe  the  types  of  tests,  then 
the  analytical  determination  of  paths  from  the  program  flowchart.  Next  we 
will  tell  about  driver  systems  and  associated  algorithms,  and  the  results  and 
limitations  of  the  system.  Finally  we  conclude  by  considering  advantages  and 
disadvantages  of  the  model  and  propose  future  directions  in  the  research 
effort. 


*Note  that  forced  testing  may  result  in  program  outputs  which  differ  from 
those  produced  by  natural  testing;  however,  these  can  be  readily  identified 
by  the  tester. 
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2.3.3  Types  of  Testa 

We  shall  begin  with  a formal  definition  of  various  types  of  testing 
strategies.  We  shall  note  that,  in  devising  a classification  scheme  for 
testing  models,  it  is  natural  to  desire  that  it  correspond  to  an  increasing 
(or  decreasing)  hierarchy  of  thoroughness,  and  difficulty.  Clearly,  the 
upper  range  of  our  numerical  scheme  should  correspond  to  an  exhaustive  test. 
At  the  lower  end  of  the  range  we  will  require  only  that  each  instruction  be 
executed  at  least  once. 

We  might  liken  the  types  of  tests  to  the  test  procedures  which  an  owner 
might  apply  to  test  a new  car  he  has  just  purchased  from  a d-iler.  The  first 
and  most  rudimentary  check  would  be  to  compare  the  list  of  accessories  or- 
dered with  the  delivered  list  on  the  car  window,  and  to  see  if  these  are 
present  and  work.  For  example,  the  owner  might  check  to  see  that  he  got  an 
AM/FM  radio,  and  that  it  works  on  both  AM  and  FM;  that  he  received  a V-6 
engine  and  not  a straight  six  or  a V-8  and  that  the  engine  starts,  that  the 
hood  lamp,  glove  box  lamp,  and  trunk  lamp  were  installed  and  work,  etc. 

This  checklist  type  test  would  be  of  the  lowest  test  level.  At  the  other 
extreme  would  be  functional  testing,  for  example,  the  use  of  the  auto  for 
three  months.  However,  in  between,  he  would  try  many  things  during  his 
first  week  of  driving:  drive  the  car  up  a hill  with  and  without  the  air 
conditioner  on,  try  the  heater  on  a cool  day  and  the  air  conditioner  on  a 
hot  one  (or  alternate  the  two  functions),  accelerate  from  rest  to  60  mph, 
and  try  a panic  stop,  and  so  on. 

Thus,  the  philosophy  for  test  classification  which  we  will  use  applies 
to  product  testing  in  general;  however,  the  specific  details  will  apply  to 
software  in  particular. 

a)  Completeness  and  Continuity  Checking  - Type  0 

This  type  of  testing  requires  that  each  instruction  be  exercised  at 
least  once. 

Intuition  tells  us  that  in  testing  a mechanism  one  basic  principle  is  to 
try  and  exercise  the  parts.  In  the  case  of  a program,  such  a test  is  very 
much  expedited  by  a modern  compiler  which  produces  counts  of  the  number  of 
times  each  statement  is  executed.  Type  0 test  is  a necessary  but  not  suffi- 
cient condition  for  thorough  testing  of  the  program.  In  fact,  when  such  a 
test  is  employed,  one  often  finds  design  flaws.  For  example,  it  is  sometimes 
Impossible  to  reach  a section  of  code,  and  upon  detailed  investigation,  we 
find  that  an  error  was  corrected  by  inserting  a patch  to  bypass  a block  of 
code;  however,  such  block  was  never  removed  and  just  remains  inert. 

We  will  call  this  lowest  level  type  of  test  a Type  0 test.  Obviously, 
it  be  performed  at  the  module  level  as  well  as  at  the  system  integration 
lcvd;  however,  it  is  more  common  to  allow  the  individual  coder  (or  tester) 
freedom  at  the  module  stage  to  proceed  as  he  wishes . Thus , much  of  our 
definition  of  test  types  is  more  applicable  to  integration  testing. 
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One  of  the  problems  In  testing  a program  at  a level  higher  than  zero  Is 
the  dependence  between  the  data  and  the  decider  predicates  - expressions 
which  control  the  branching  of  an  IF- THEN- ELSE  or  DO  WHILE  instruction  - In 
the  program.  Intuition  tells  us  that  once  we  have  completed  a check  list  for 
a Type  0 test,  we  should  next  test  all  paths  in  the  program.  If  we  use  a 
flowchart  as  our  program  abstraction,  we  can  define  all  paths  of  the  chart; 
however,  it  is  unfeasible  to  find  by  manual  analysis  all  possible  executable 
paths  in  most  programs.  Thus  an  automated  tool  is  highly  desirable. 

In  the  solution  to  the  problem  of  constructing  a program  testing  tool, 
it  is  convenient  to  define  two  classes  of  path  tests:  force  execut Ion-Type 
1,  and  natural  execution- Type  2.  By  natural  execution  we  mean  that  the  test- 
er (human  or  machine)  reads  the  decider  predicates,  computes  whether  they  are 
true  or  false  based  on  the  current  values  of  the  program  variables,  and 
branches  left  or  right  accordingly.  This  concept  applies  as  well  to  3-way  or 
multi-way  branching,  because  such  can  always  be  expressed  by  2-way  branches; 
modern  IF- THEN- ELSE  constructs  express  indeed  this  fact  that  a condition  is 
either  true  or  false. 

The  execution  time  of  a program  is  often  largely  devoted  to  the  repeti- 
tive execution  of  DO  loops  within  the  program.  However,  the  philosophy  of  a 
forced  test  is  to  execute  all  paths  which  only  include  two  executions  of  a 
program  loop  at  most.  Thus,  we  must  invent  a technique  to  ensure  that  each 
DO  loop  is  traversed  no  more  than  twice.  We  also  know  from  experience  that 
many  errors  are  committed  when  we  exit  from  a loop.  Thus,  we  define  the 
forced  execution  of  a DO  loop  as  testing  the  loop  twice,  once  for  the  first 
execution  and  again  for  the  last  execution.  Methods  of  forced  execution  of 
paths  and  of  DO  loops  are  discussed  in  Section  2.3.5. 

Another  question  which  is  relevant  to  force  traversal  methodology  is 
whether  or  not  input  data  have  to  be  supplied  at  execution.  Obviously,  data 
which  affect  the  flow  of  control  are  not  needed  and  can  be  omitted;  however, 
other  data  types,  such  as  operands  In  arithmetic  expressions,  may  profitably 
be  submitted  if  the  user  is  interested  in  such  testing.  Therefore,  it 
essentially  depends  whether  the  user  of  force-traversal  testing  merely  wishes 
to  check  the  control  flow  and  path  structure  for  which  no  input  data  are 
necessary.  If  he'd  like  to  test  for  consistency  of  results,  he  must  supply 
input  values.  At  any  rate,  a Type  1 driver  can  run  without  any  input  data 
whatsoever;  and  it  is  this  fully  automatic  mode  of  operation  that  renders  the 
model  so  attractive.  The  fact  that  some  program  variables  may  contain  mean- 
ingless quantities  is  but  a natural  limitation  of  this  type  of  model  and  can 
be  solved  only  by  an  escalation  to  a higher  testing  model;  nevertheless,  the 
present  strategy  greatly  enhances  the  user's  access  to  thorough  testing  of 
programs . 
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Based  on  this  discussion,  a force  execution  of  psths  is  called  s Type  1 

test. 

c)  Bxauative  Testing  - Types  3 and  4 

Similarly,  we  define  here  two  types  of  exhaustive  tests.  A Type  3 test 
is  an  exhaustive  test  for  a system  where  input  sequence  and  initial  condi- 
tions are  fixed.  A Type  4 test  is  an  exhaustive  test  for  a system  where 
either  the  initial  conditions,  or  the  input  sequence,  can  change  during  pro- 
gram execution. 

In  Table  1 we  suamarize  the  test  class  definitions  which  we  have  evolved, 
and  discuss  three  typical  "in  between"  classifications. 


TYPE  0 All  Instructions  in  code  executed  at  least  once  (check  j 

list). 

TYPE  0.5  Many  paths  force  executed  at  least  once. 

TYPE  1 All  paths  force  executed  at  least  once  (simulated  100Z 
coverage) . 


TYPE  1.5  All  paths  force  executed,  some  naturally  executed. 

TYPE  2 All  paths  naturally  executed  at  least  once  (path  coverage 
100Z).  This  test  is  not  unique. 

TYPE  2.5  All  paths  naturally  executed  for  many  values  of  input 
parameters. 

TYPE  3 All  paths  naturally  executed  for  all  values  of  input  para 
meters  (exhaustive  test). 


TYPE  4 All  paths  naturally  executed  for  all  values  of  input  para- 
meters, all  sequences  of  inputs,  and  all  combinations  of 
initial  conditions  (exhaustive  test  for  multiprocessing, 
multiprogramming,  and  real  time  systems  with  non- fixed 
input  sequence). 


TABLE  1 — Classification  of  Tests 


2.3.4  Analytical  Determination  of  Program  Paths 


In  thia  sactlon  we  analyze  the  relationship  between  the  number  of  deci- 
der predicates  In  a loopless  program  and  the  number  of  program  paths. 

First,  an  upper  and  lower  bound  are  determined.  Then  a decomposition 
procedure  Is  explained  and  an  example  Is  given  which  shows  how  all  possible 
paths  In  a program  flowchart  can  be  Identified  from  Its  structure. 

a)  Bounds  on  the  Humber  of  Paths  In  a Loopless  Program 


The  Important  properties  of  flow  charts  are: 

(1)  the  number  of  decision  elements  (deciders); 

(2)  the  number  of  points  where  two  or  more  feed  forward  branches 
meet  (merges) ; 

(3)  the  number  of  points  where  a feed  forward  path  meets  a 
feedback  path  and  creates  a loop. 

For  simplicity  we  assume  that  the  graph  has  no  loops  and  we  bound  the 

number  of  paths  by  the  number  of  deciders  and  merges.  In  Fig.  1(a)  we  show  a 

graph  with  m deciders  and  m merges.  Each  decider-merge  pair  furnishes  two 
paths  and  by  virtue  of  the  chain  structure  we  see  that  the  number  of  paths 

for  the  total  graph  is  simply  the  product  of  each  subgraph  path,  i.e.,  2* 
paths.  In  Fig.  1(b)  we  portray  a structure  with  n deciders  and  one  merge. 

The  first  decider  creates  two  paths.  The  next  decider  takes  up  one  of  the 
paths  as  Its  Input  and  creates  two  new  paths.  Thus,  there  are  nfl  paths  In 
this  graph.  As  an  example  of  the  application  of  these  bounds,  consider  a graph 

with  13  deciders.  The  number  of  paths  in  such  a graph  Is  between  14  and 

8192.  Fortunately,  the  number  of  paths  In  a program  is  usually  close  to  the 
lower  bound. 


Fig.  1(a).  An  upper  bound.  Fig.  1(b).  A lower  bound. 

FIGURE  1.  Graphs  illustrating  bounds  on  the  number  of  paths  In  a flow  chart. 
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b)  Manual  Determination  of  the  Humber  of  Paths  From  a Flowchart 

If  a program  la  written  In  structured  top-down  form  or  any  other  modular 
form,  the  program  can  easily  be  divided  Into  Independent  subgraphs.  In  the 
case  of  a nonstructured  design,  subdivision  can  still  be  performed  with 
analogous  techniques . 

In  performing  subdivisions,  the  elementary  substructures  given  In  Fig.  2 
are  encountered.  In  Fig.  2(a)  the  number  of  paths  in  the  program  between 
point  A and  stop  or  stops  Is  denoted  by  N^.  Clearly  this  number  Is  the  sum 

of  the  number  of  paths  attached  to  the  left  hand  branch  N_  and  those  attached 

D 

to  the  right  hand  branch  N^,.  In  Fig.  2(b)  the  branch  merge  structure  multi- 
plies the  number  of  paths  seen  at  point  B by  2.  In  the  case  of  Fig.  2,  we 
end  up  with  two  equalities  at  the  merge  as  shown. 


Fig.  2(a).  Branch.  Fig.  2(b).  Branch-Merge  Fig.  2(c).  Merge. 

FIGURE  2.  Elementary  graph  sub-structures. 


We  will  Illustrate  the  calculation  of  the  number  of  paths  In  a program 
with  n conditional  jumps.  From  the  previous  discussion,  we  know  this  number 

to  be  In  the  range  (n+1,  2 ).  Let  us  now  consider  the  following  example: 

Assume  that  the  computer  Is  to  determine  the  winner  of  a card  game  In 
which  player  A is  dealt  two  cards:  Al,  A2,  and  similarly  player  B is  dealt 
two  cards:  Bl,  B2.  If  the  players  have  any  pairs,  the  highest  pair  wins, 
otherwise  the  player  with  the  highest  card  wins.  If  both  players  have  the 
same  high  card,  then  the  winner  Is  the  player  with  the  highest  second  card. 
Identical  hands  with  or  without  pairs  are  ties.  A flow  chart  for  this  pro- 
gram Is  given  In  Figure  3.  There  are  13  deciders,  and  each  branch  Is  lden- 
tlfled  with  letters  A,  A',  B,  etc.  The  flowchart  is  decomposed  In  submodules 
as  In  Figure  2.  The  simple  algebraic  relationships  which  can  be  derived  are 
In  Table  2.  All  paths  are  identified  and  taken  Into  account  one  by 
one;  the  final  computation  for  this  structure  with  13  deciders  yields  100 
paths. 
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Algebraic  Relationship  Number  of  Paths 


2xNa, 


N =Na+Na,  -2xNa, 

NA'  " NB  + NB'  = 2 X NB* 

V = HC  + nd 

NC  " NE  + NE’  = 1 + NE' 

NE'  = NF  + NF'  = 1 + 1 " 2 

ND  " N0  + NG*  = 1 + NG' 

ng-  - nh  + nh-  = 1 + nh’ 

NH*  'V1!''2*  NI* 

Nr  * Nj  + NJt  - 2 x NJt 

Nj,  = Nr  + Nk,  - 1 + Nk, 

Nk,  - M1  + IL,  - 1 + Ilt 

NL*  - Nm  + V " 1 + NM* 

nm«  " nn  + V “ 1 + 1 - 2 


U x Nb, 

U x (Nc  + Nd) 

U x ((1  + Ne.)  + Nd) 

4 x (3  + Bp) 

4 x (3  + 1 + Ng, ) 

4 x (4  + 1 + NRI ) 

4 x (5  + 2 x lJIt) 

Ux  (5  + l*xNJt) 

4 x (5  + 4 x (1  + Nk,)) 

U x (5  ♦ U x (1  ♦ 1 ♦ Hl,)) 

4 x (5  + 4 x (2  + 1 + Nm,)) 

4 x (5  +4  x (3  + 2))  ■ 100 


TABLE  2.  Calculation  of  the  number  of  paths  in  the  flowchart  of  Figure  3 


2.3.5  Driver  Systems 


Ve  will  now  introduce  the  practical  implementation  of  Type  1.5  driver 
systems.  Such  drivers  force  the  traversal  of  a given  subject  program  through 
all  its  paths. 

Recall  that  if  ve  naturally  execute  a subset  of  all  program  paths,  then 
we  refer  to  such  a test  as  being  between  Type  1 and  Type  2.  Similarly,  in 
most  cases  forced  execution  will  coincide  with  natural  traversal  of  some 
paths  and  forced  traversal  of  the  remainder;  consequently,  we  describe  the 
drivers  discussed  here  as  resulting  in  Type  1.5  tests. 

The  design  of  drivers  has  evolved  through  several  phases  during  the 
present  research  on  testing  models.  The  most  obvious  technique  for  complete 
path  traversal  is  referred  to  here  as  an  "upper  bound"  driver  and  is  des- 
cribed in  section  2.3.5(a).  Such  a design,  as  it  will  be  shown,  achieves  the 
goal  of  automated  testing  at  a high  penalty.  Further  considerations  and  re- 
finements of  the  problem,  namely,  the  realization  of  an  algorithm  for  path 
analysis,  have  led  to  the  implementation  of  a system  of  programs  which  con- 
stitute the  whole  driver  system.  These  will  be  described  in  a future 
technical  report  on  this  topic. 

a)  An  "Upper  Bound"  Driver 

The  system  described  here  was  a first  attempt  to  implement  a driver  to 
force  the  execution  of  a PL/1  program  under  test,  from  now  on  referred  to  as 
the  subject  program. 

The  subject  program  is  written  in  standard  PL/1  with  no  restrictions; 
there  are  only  a few  precautions  the  progrananer  must  take  in  designing  his 
code: 


o the  total  number  of  IF-statements  and  repetitive  DO-groups,  herein 
called  NTESTS,  must  be  supplied  on  a data  card; 


o each  statement  of  the  form 

o each  statement  of  the  form 
must  be  written  as 


IF  cond  . . . must  be  written  as 
IF  F(cond)  . . . 

DO  I-limitl  TO  llmlt2  BT  Increment 
DO  I”GL(limitl,  limit2)  TO  GH  BT  Increment 


o each  statement  of  the  form 
must  be  written  as 


DO  WHILE (cond) 

DO  WHILE (H (cond)) 


o functions  and  subroutines  must  be  Internal 


The  deck  of  the  subject  program  is  then  simply  Inserted  within  the  deck 
of  the  driver  program  at  an  appropriate  location.  The  driver  exercises  all 
paths  through  several  runs. 


Th«  driver's  mode  of  operation  is  simply  based  on  the  fact  that  the 

NTESTS 

upper  bound  on  the  number  of  possible  paths  is  2 (see  section  2.3.4  a)) 

The  driver  program  will  internally  construct  a binary  number,  called  control 
word,  with  NTESTS  bits,  whose  Initial  value  has  all  bits  set  to  0;  this 
number  is  increased  by  1 at  each  run  during  execution,  until  the  control  word 
has  all  bits  set  to  1. 

During  each  of  the  runs,  function  F (as  well  as  GL,  GH  and  H)  replaces 
the  value  of  the  condition  with  the  corresponding  bit  from  the  control  word. 

Functions  GL  and  GH  cause  a DO-group  with  index  variable  to  be  executed  once 
with  the  initial  value  of  the  index  (bit“0)  and  once  with  the  final  value 
(bit”l) ; function  H causes  execution  of  a DO  WHILE  group  exactly  once  in  any 
case. 

NTESTS 

Since  there  are  2 possible  distinct  values  of  the  control  word,  J 

NTESTS 

there  will  be  exactly  2 runs  of  the  subject  program;  therefore  coverage 

of  all  possible  paths  is  mathematically  guaranteed,  i.e. , each  path  will  be 
covered  at  least  once.  Hence  the  goal  of  automated  force  traversal  is  fully 
achieved  with  this  simple  strategy. 

Because  the  number  of  paths  in  a program  is  usually  closer  to  the  lower 

bound  NTESTS+1  than  to  the  upper  bound  2*^*^  there  will  be  often  a large 
number  of  runs  which  do  not  represent  any  existing  paths.  For  Instance,  the 
flowchart  of  Fig.  3 has  13  deciders  but  only  100  paths;  hence  8092  runs  are 
wasted  with  this  strategy.  Furthermore,  since  the  number  of  runs  increases 
exponentially  with  the  number  of  predicates,  the  running  cost  of  such  a 
driver  becomes  prohibitive  very  soon,  even  for  medium  size  programs. 

This  problem  can  be  overcome  by  the  derivation  of  the  path  structure  of 
a program  from  its  code,  that  is,  from  static  analysis.  This  strategy  will 
be  described  in  the  following  sections. 

b)  A Type  1.5  Driver 

The  complete  driver  system  is  shown  in  Figure  4.  It  has  a section  for 
static  path  analysis,  one  for  code  translation  and  one  for  dynamic  testing. 

At  the  left  hand  side  in  the  picture  one  recognizes  the  execution  of  the 
driver  programs  from  files  located  in  the  middle  of  the  picture.  An  input 
program  to  be  tested,  referred  from  now  on  to  as  the  subject  program,  enters 
the  path  analyzer,  which  determines  the  program  paths  and  saves  their 
representation  as  binary  path  descriptors,  along  with  a copy  of  the  subject 
program. 


The  copy  of  the  subject  program  undergoes  some  modifications  by  the 
translator  program.  This  translator  program  modifies  conditional  branches, 
loop  constructs  and  Includes  the  program  in  a large  loop.  This  inclusion 
allows  repeated  executions. 

Eventually  the  modified  subject  program  reaches  the  execution  stage 
through  all  its  paths,  as  determined  by  the  binary  path  descriptors,  and  the 
output  of  the  driver  is  produced. 


17 


FIGURE  U.  The  system  of  the  driver  programs 
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Phase 


read  input  program 
from  input  file 


outputs  of  path 

analyzer  created  / 
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modified  subject 
program  produced 


JCL  causes  operating 
system  to  execute 
modified  subject 
program  through  all 
its  paths 


and  their  operation. 
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e)  The  Algorithm  for  Path  Analysis 

Labeling  of  Paths : We  shall  usa  the  convention  to  label  the  "true" 
branch  of  a conditional  statement  with  a "l",  and  the  "false"  branch  with 
e "0",  as  seen  In  Figure  5.  In  this  way.  It  Is  possible  to  uniquely  label  a 
path  In  a given  program  with  a binary  path  description,  as  shown  In  Figure  6. 


l 

false  I true 

| * <£> * — i 

o 1 


FIGURE  5.  Labeling  of  branches. 


FIGURE  6.  Labeling  of  path  10011. 

Algorithm  for  Finding  All  Possible  Paths:  We  will  first  show  that 
It  Is  possible  to  determine  all  possible  paths.  He  will  start  with  the  path 
analysis  for  a program  without  any  repetitive  DO  constructs.  Since  each  path 
Is  uniquely  defined  by  a binary  Integer,  called  here  path  descriptor,  the 
problem  of  finding  each  path  In  a given  program  Is  analogous  to  the  problem 
of  finding  the  set  of  binary  Integers  associated  with  the  path  structure  of 
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that  program.  Because  sets  of  binary  quantities  can  be  expressed  by  regular 
expressions,  we  propose  an  algorithm  which  constructs  a regular  expression 
whose  associated  set  contains  the  values  of  control  binary  words  correspond- 
lng  to  paths.  Only  tha  operation  + (expressing  union  In  the  sssodated  set) 
and  concatenation  will  be  needed.  The  expression  Is  recursively  defined  as 
being  always  binary,  i.e. , It  contains  two  terms  separated  by  +;  a term  Is 
the  symbol  1,  or  0,  or  an  expression;  concatenation  of  expressions  forms  an 
expression. 

The  algorithm  scans  a PL/1  program  In  search  of  IF-THEN-ELSE  constructs, 
and  operates  accordingly  to  the  following  rules: 

1)  each  IF  opens  a left  parenthesis,  (,  and  initiates  an  expression 

2)  esch  THEN  corresponds  to  a 1 

3)  each  ELSE  corresponds  to  a 0,  and  since  It  matches  a previous  THEN, 

a + is  Inserted  at  that  level 

A)  if  no  matching  ELSE  Is  present.  It  Is  assumed  to  be  there  and  +0  Is 
therefore  added 

5)  each  balanced  expression,  consisting  of  1,  +,  0,  closed  at  Its 
level,  causes  closure  with  right  parenthesis  at  that  level. 

2.3.6  Examples 

EXAMPLE  1. 

Consider  the  flowchart  of  Figure  7.  This  chart  Is  implemented  by  the 
program  segment 

IF  cond  THEN  s 
ELSE  s 
IF  cond  THEN  8 
IF  cond  THEN  s 
ELSE  a 

The  algorithm  constructs: 

(1+0) (1+0) (1+0) , with  rules 
12  3512  A5123  5 

Computation  of  the  regular  expression  yields. 

111,011,101,001,110,010,100,000 

l.e.,  the  eight  possible  paths. 
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EXAMPLE  2 


The  flowchart  of  Figure  8 translates  into: 

IF  cond  THEM  IF  cond  THEM  IF  cond  THEM  e 

ELSE  s 

ELSE  IF  cond  THEM  s 
ELSE  s 

ELSE  IF  cond  THEM  IF  cond  THEM  s 

ELSE  s 

ELSE  IF  cond  THEN  s 
ELSE  • 


Regular  expressions: 

(1 (1 (1+0 )+0 (1+0) )+0 (1 (1+0)+0 (1+0) ) ) , rules : 

121212  35  312  35  31212  35  312  35 
expressing  the  eight  paths 

in,  no,  ioi,  100,011, 010,001,000 


EXAMPLE  3 

The  flowchart  of  Fig.  9 Is  programed  by 

IF  cond  THEM  s 

ELSE  IF  cond  THEM  s 

ELSE  IF  cond  THEM  s 

ELSE  IF  cond  THEN  s 
ELSE  s 

Regular  expression: 

(1+0 (1+0 (1+0 (1+0)))),  rules: 

12  312  312  312  35 

which  gives 

1,01,001,0001,0000 
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EXAMPLE  4 


The  flowchart  of  Fig.  10  la  realized  by 

IF  cond  THEM  DO; 

IF  cond  THEN  IF  cond  THEN  8 

ELSE  s 

ELSE  IF  cond  THEN  s 
ELSE  a 

IF  cond  THEN  s 

END; 

ELSE  DO; 

IF  cond  THEN; 

ELSE  s 

END; 

Regular  expression: 

(1(1(1+0)+0(1+0)) (1+0)40(1+0) , rules: 

121212  35  312  35  12  45  312  35 

yielding 

1111 , 1110 , 1101 , 1100 , 1011 , 1010 , 1001 , 1000 ,01,00 

In  Example  4,  the  path  structure  Is  complicated  by  the  presence  of  DO 
groups  within  the  THEN  and  ELSE  clauses.  When  this  occurs,  construction  of 
the  regular  expression  temporarily  halts,  and  the  algorithm  calls  Itself 
recursively  for  that  DO  module. 

This  algorithm  has  been  tested  with  a program  written  in  LISP  which  was 
then  translated  Into  a PL/1  program. 

A second  algorithm  solves  the  regular  expression  and  finds  all  the  ele- 
ments of  Its  associated  set;  this  number  Is  the  mmfter  of  all  possible  paths, 
hence  the  algorithm,  as  an  extra  bonus,  enumerates  all  paths  of  a program. 

The  complete  analyzing  program  embodies  these  algorithms.  It  starts  by 
reading  In  the  subject  program,  card  by  card.  This  is  stored  as  a character 
string  in  main  memory  and  is  saved  on  an  external  file,  called  ORIG.  Control 
is  then  passed  to  the  section  performing  the  scanning  and  computing  the  set 
of  binary  control  words. 

The  scanning  mechanism  is  built  around  two  PL/1  procedures,  CAR  and  CDR, 
which  respectively  return  the  head  and  the  tail  of  a given  string.  By  head 
we  mean:  any  PL/1  operator  (such  as  *,**, — , ),  any  PL/1  separating  char- 

acter (such  as  ;,  :,  (),  )or  any  string  separated  by  an  operator,  by  a sepa- 
rating character  or  by  blank,  or  any  quoted  string  of  counent;  by  tall  we 
mean  the  string  without  its  head.  Hence  CAR  is  capable  of  correctly  picking 
up  keywords  in  portions  of  statements  such  as  ;IF(,*/THEN/. , L2:DO;,  but  will 
not  pick  them  up  In  THEN 1-5,  'A  STRING  IF  NEEDED' ,/* THEN  A COMMENT*/. 
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The  analyzing  program  produces  the  set  of  binary  path  descriptors, 
saved  on  external  file  REGEXP,  which  will  control  the  test  portion. 

2.3.7  Results 


In  this  section  ve  shall  discuss  the  usefulness  of  Type  1.5  drivers  for 
forced  testing,  comparing  their  use  with  other  widely  used  testing  strategies, 
and  describe  the  efficiency  and  deficiencies  of  the  present  Implementation. 

a)  Automatic  Analysis  and  Forced  Execution 

Ve  have  shown  in  Section  2.3.4(b)  that  even  in  a simple  program  like 
the  one  of  Fig.  3,  path  Identification  and  enumeration  Is  not  a trivial 
problem,  but  time  consuming  and  error  prone.  In  such  cases,  the 
algorithm  described  In  Section  2.3.5(c)  has  great  advantages,  because  it 
not  only  Identifies  all  paths  and  properly  labels  them,  but  the  path 
analyzer  program  saves  all  binary  path  descriptors.  Hence  It  Is  not 
necessary  to  spend  time  in  designing  data  sets  to  exercise  all  paths  (with 
the  result  of  not  covering  them  all,  as  it  generally  happens).  The  driver 
system  takes  the  Input  program;  the  analyzer  creates  the  descriptor  for 
each  run  or  path,  the  translator  modifies  the  code,  and  the  modified  pro- 
gram runs  100  times  through  all  Its  paths. 


b)  Comparison  Between  Manual  and  Automatic  Testing 

He  established  a timetable  to  compare  results  and  time  spent  by  testing 
with  Input  data  and  with  the  driver  system.  He  began  by  saying  that  it  took 
30  minutes  to  design  a successfully  compiled  program.  He  then  tested  the 
program  manually  and  automatically.* 

Manual  Testing  with  Data 

• design  of  a data  set  to  exercise 
some  paths:  10  minutes 

a running  of  the  program  through 
these  paths:  0.01  minutes 

a results:  program  is  OK 


Automatic  Testing  with  the  Driver 

a path  analyzer:  1.13  min,,  450k 
core. 

a translator:  0.27  minutes 
a translated  subject:  all  paths 
a results:  one  bug  was  discovered 


(The  bug  appeared  in  the  form  of  two  contradictory  outputs;  an  erroneous 
ELSE  clausa  was  subsequently  fixed  In  the  subject  program.) 

It  appears  therefore  that  no  particular  penalty  In  running  time  has  to 
be  paid  to  let  a program  run  through  all  Its  paths.  Recall  in  fact  that 
in  general  the  number  of  paths  will  be  closer  to  the  lower  bound  than 
to  the  upper  bound.  Moreover,  program  with  repetitive  DO-loops  may  very 
well  run  faster  through  the  driver  than  with  some  manual  testing,  because 
extra  saving  In  running  is  achieved  by  not  letting  loops  run  more  than 
twice.  This  saving  may  become  very  consistent  In  a program  with  many 
nested  loops. 


* These  operations  were  carried  out  on  the  IBM  360/65  computer  at  the  Poly* 
technic  Institute  of  New  Tork,  Brooklyn,  NT  11201. 


Since  it  is  not  necessary  to  design  sny  testing  data,  the  programmer 
is  saved  time  and  effort  for  manual  debugging.  There  is,  moreover,  no 
guarantee  that  testing  with  data  achieves  the  result,  as  this  case  shows 
the  erroneous  path  was  accidentally  never  reached  with  testing  data. 

The  path  analyzer  program,  however,  is  not  very  efficient  at  present  and 
requires  prohibitive  amounts  of  computer  memory,  as  discussed  in  the  next 
section. 

c)  Efficiency  of  the  Package 

The  path  analyzer  program  is  a literal  implementation  of  the  recursive 
algorithm  described  in  Section  2.3.5(c).  As  such,  it  is  slow  and  inefficient 
and  requires  a large  amount  of  computer  memory.  This  is  due  to  the  fact 
that  each  time  a recursive  procedure  which  is  still  open  calls  Itself 
either  directly  or  indirectly,  it  reallocates  a new  space  in  the  main 
storage.  In  PL/1,  unfortunately,  a string  of  maximum  length  is  allocated 
even  if  the  string  is  declared  as  variable  length.  As  a result,  the 
path  analyzer  wastes  a large  number  of  bytes.  This  wasted  memory  is  recu- 
perated only  when  the  last  recursive  call  terminates,  at  which  point  the 
recursion  stack  begins  to  pop. 

The  problem  has  in  fact  been  avoided  by  implementing  the  algorithm  for 
path  analysis  in  LISP,  a language  Ideally  suited  for  recursive  function  and 
string  (l.e. , list)  manipulations.  No  prohibitive  memory  is  needed,  since 
the  pairing  by  pointer  mechanism  uses  space  only  when  needed.  However,  the 
LISP  Interpreter  is  not  widely  available,  and  is  in  general  very  slow  com- 
pared to  compilers  for  other  high  level  languages. 

One  solution  to  the  inefficiency  of  the  analyzer  we  are  considering 
would  be  to  still  keep  the  recursive  algorithm  in  PL/1.  However,  Instead 
of  storing  in  the  stack  the  character  string  resulting  from  an  intermediate 
computation,  we  would  operate  always  only  on  one  copy  of  the  string  and 
save  rather  the  present  scanning  location,  i.e.,  a one-word  pointer.  Hence 
we  would  achieve  a saving  of  32K-bytes  versus  one  word,  which  is  close  to 
an  order  of  magnitude  of  A.  A drastic  saving  in  memory  space  would  be 
realized,  and  simultaneously  some  techniques  to  speed  up  the  algorithm 
could  be  added. 

The  tr«fn«lator  program  is  essentially  built  around  string  manipulating 
built-in  procedures,  and  is  therefore  fast.  No  doubt,  however,  it  could 
still  be  further  optimized. 

The  translated  subject  program  will  run  through  all  its  paths.  In  the 
previous  section  we  point  out  that  this  may  be  achieved  at  the  price  of  a 
high  penalty  in  running  time.  It  is  worth  mentioning,  however,  that  even  if 
this  were  so,  the  advantage  provided  by  an  automatic  mechanism  which  is 
guaranteed  to  exercise  all  paths  in  a subject  program  outwelghts  some  possi- 
ble disadvantages  of  extra  running  time. 
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d)  Present  Restrictions  on  the  Subject  Program  and  Limitations  of  the 
Driver  System 

Our  system  Implements  a driver  system  In  order  to  force  execution  of  a 
subject  program  written  In  PL/1  through  all  Its  possible  paths.  Almost  all 
PL/1  can  be  used  In  the  design  of  the  subject  program,  with  a few  restric- 
tions. Some  depend  on  our  computer  system  and  some  were  Introduced  for 
ease  of  design  of  the  system.  Some  are  tolerable  or  even  welcome  when  they 
encourage  good  programming  style.  Some  are  undesirable  and  will  hopefully 
be  removed  in  the  next  future. 

As  shown  in  Section  2.3.3(b),  the  driver  will  force  execution  of  paths 
that  could  never  be  reached  by  natural  execution,  hence  capturing  non- 
existent errors.  This  Is  a flaw  of  a Type  1 testing  model  and  could  con- 
ceivably only  be  solved  by  a higher  model.  It  is  however,  unclear  whether 
the  determination  of  Impossible  paths  can  be  solved  at  all.  Assume  that 
within  certain  lines  of  code,  an  algorithm  is  called  for  the  computation 
of  one  of  the  values  used  In  the  next  decider.  At  testing,  it  Is  unknown 
whether  the  algorithm  will  terminate  or  not  (i.e.,  the  halting  problem). 

This  example  shows  that  It  is  in  general  impossible  to  exercise  a path, 
even  at  level  1.  Our  driver,  in  fact,  discontinues  a path  if  it  takes 
longer  than  a certain  time,  thus  putting  a bound  in  order  to  avoid  solu- 
tion to  the  halting  problem.  Possibly,  only  those  Impossible  paths 
depending  on  assignments  of  known  values  to  the  predicate  variables  can 
be  identified,  and  possibly  only  a subset  of  them  could  be  found  by  static 
analysis.  In  the  meantime,  therefore,  we  shall  regard  this  problem  as  not 
highly  important;  the  user  can  in  fact  quickly  identify  these  paths  from 
the  driver's  output. 

We  will  now  consider  a few  implementation  restrictions.  Our  version  of 
PL/1  does  not  allow  strings,  that  is,  subject  programs  with  more  than 
32767  bytes  (about  400  cards).  Without  modifying  our  local  definition  of 
PL/1,  this  could  again  be  solved  by  a swapping  mechanism,  which  would  call 
from  disk  the  different  sections  of  a program  under  analysis  or  translation. 

Ease  in  program  design  of  the  analyzer  requires  the  exclusion,  in  the 
subject  program,  of  any  variable  or  identifier  named  IF,  THEN,  ELSE,  DO,  END, 
BEGIN,  TO,  BT  and  WHILE.  Similarly,  multiple  clause  DO-groups  (clauses  are 
separated  by  commas),  and  multiple  closure  with  a labeled  END  — all  this  is 
allowed  in  PL/1  — have  to  be  avoided.  However,  use  of  these  PL/1  features 
often  causes  confusion,  and  often  programmers  are  independently  encouraged 
not  to  use  them. 

Moreover,  although  it  is  a minor  drawback  that  the  driver  cannot  handle 
GO  TO' a — they  can  always  be  avoided  in  structured  programs,  and  almost  al- 
ways with  profit  — it  is  more  serious  that  it  cannot  cope,  at  this  stage  of 
development,  with  branching  to  subroutines.  A mechanism  could  be  added  in 
order  to  allow  scanning  to  Jump  to  the  subroutine  location,  saving  the  cur- 
rent scanning  location,  to  which  to  return  upon  exit  from  the  procedure. 

This  is  conceptually  simple  but  trill  involve  some  tedious  readjusting  of  the 
driver's  code;  however,  we  plan  to  do  this  in  the  near  future. 
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It  Is  unclear,  however,  how  a static  path  analyzer  could  cope  with  re- 
cursive procedures,  because  the  path  structure  of  a recursive  program  Is  un- 
known before  execution  time  — In  fact,  no  graph  or  flowchart  representation 
exists  for  a recursive  algorithm.  Let  us  just  mention  the  fact  that  any 
recursive  function  can  always,  though  sometimes  painfully,  be  made  non- 
recursive;  besides,  recursive  programs  are  rarely  found  in  the  real  world  of 
programming . 

Except  for  its  inability  to  handle  procedures,  the  driver  system  is 
therefore  general  enough  to  accept  any  well  written  PL/1  program  to  be 
tested. 


2.3.8  Summary  and  Conclusions 

Ue  conclude  by  describing  a practical  application  for  this  implementa- 
tion of  a Type  1.3  testing  model. 

The  described  driver  system  can  be  integrated  into  an  operating  system. 

A program  would  initially  be  compiled  to  catch  syntax  errors  and  upon 
successful  compilation.  It  would  be  submitted  to  the  driver  system  for  forced 
execution.  Therefore,  another  set  of  errors,  appearing  at  execution  time, 
could  be  eliminated  prior  to  testing  with  real  data.  Alternatively,  a 
strategy  intermingling  natural  and  forced  execution  could  be  implemented. 

We  would  like  to  recall  once  more  that  our  effort,  although  directed 
mainly  toward  PL/1  programs,  can  be  extended  to  other  programing  languages 
as  well.  We  hope  therefore  that  our  driver  techniques  represent  a step 
toward  automatic  debugging. 

As  a final  conclusion,  we  notice  that: 

1.  Although  the  area  of  testing  is  a difficult  one,  we  have  developed 
several  quantitative  models  and  approaches  to  aid  research  progress  in  the 
field. 

2.  A quantitative  way  of  describing  and  categorizing  different  types  of 
tests  has  been  developed  which  may  aid  discussion  and  characterization  of 
tests. 

3.  A system  of  algorithms  to  perform  automatic  Type  1.3  testing  has 
been  implemented  and  described  in  detail.  It  was  shown  that  such  a driver 
is  Indeed  feasible  and  advantageous.  An  explicit  computation  of  the  number 
of  paths  in  a graph  was  carried  out  analytically.  The  same  example  has  been 
submitted  to  the  driver  system  and  run  through  the  driver.  This  Illustra- 
tion can  be  extended  to  other  examples  and  generalized.  Hence,  the  testing 
model  has  been  defined,  researched  and  fully  implemented.  Although  the  model 
has  already  proved  itself  a useful  tool,  it  is  hoped  that  It  will  clarify 
and  further  stimulate  research  in  this  area. 


* 
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path  through  the  program,  nor  teatlng  one  module  of  the  program.  To  thle 
end  the  matrix  o has  been  Introduced  so  aa  to  tie  together  these  two  views  of 
testing.  Formally,  the  Introduction  of  the  matrix  <r  might  be  said  to  des- 
cribe the  statistical  dependence  among  errors  for  different  Input  values. 

It  Is  Implicit  that  the  program  as  a whole  must  be  tested,  l.e. , separate 
modules  cannot  be  Individually  tested  In  this  model.  Several  possible  ways 
to  define  a bug  will  now  be  defined  and  compared: 

I)  The  program  has  M modules  and  bug  1 means  that  the  lt^1  module  doesn't 
work  properly.  This  will  cause  errors  only  for  those  input  values  which  cause 

the  program  to  call  the  i**1  module. 

II)  The  program  has  only  one  module  with  one  input  variable,  and,  for 
example,  bug  1 means  It  fails  for  all  positive  input  values,  but  2 means  It 
falls  for  0 Input,  and  bug  3 means  It  falls  for  all  negative  input  values. 

III)  The  program  has  M lines,  and  bug  1 means  that  the  1th  line  has  an 
error. 

iv)  Some  combination  of  the  above.  For  example,  bug  1 might  refer  to 
negative  Input  values,  bug  2 to  subroutine  B,  etc. 

These  and  other  possible  interpretations  must  be  compared  by  considering 
how  realistically  they  correspond  to  actual  programing  situations,  and  to 
the  relative  difficulty  of  gathering  data  and  statistics  which  can  be 
inserted  Into  the  formula  (1).  To  start  with,  (111)  seems  attractive 
since  all  programs  are  line  or  statement  structured.  Perhaps  one  might 
like  to  consider  that  each  line  might  have  n possible  bugs,  so  that  M - nL. 

The  big  difficulty  with  (111)  is  that  the  error  might  be  a missing  line, 
say  a variable  that  was  not  Initialised.  If  one  allows  for  m possible 
missing  lines,  then  M - nL  + m;  but  the  effect  of  a missing  and  unknown 
line  on  a particular  Input  value  Is  hard  to  anticipate.  Somewhat  the 
same  objections  can  be  made  to  (1).  In  particular,  many  additional  possi- 
ble bugs  exist  as  Incorrect  linkages  between  modules.  While  (11)  may 
appear  to  avoid  the  difficulties  just  mentioned.  It  requires  more  know- 
ledge of  what  the  true  outputs  should  be  for  all  input  combinations.  In 
other  words,  (11)  is  problem  oriented  Instead  of  being  program  oriented, 
and  this  might  make  the  gathering  (or  simply  guessing)  of  statistics  more 
difficult. 

In  addition  to  properly  interpretting  the  bugs,  Eq.  (1)  above  should  be 
simplified  If  it  la  to  be  used  in  practice.  In  the  previous  report,  (Ibid, 

Eq.  16)  a simplified  version  was  given,  but  this  Is  perhaps  too  simple.  Work 
Is  going  on  In  finding  a useful  compromise  between  these  extremes. 
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2.5  COMPLEXITY  MEASURES  by  H.  Ruston,  A.E.  Laenmel,  and  E.  Berlinger 

A nev  complexity  measure  A preliminary  report  on  complexity  measures 
is  partly  completed.  The  report  first  reviews  and  contrasts  several  of  the 
meaaures  introduced  so  far.  It  Includes:  (1)  the  measure  based  upon  divi- 
sion of  Inputs  into  classes,  with  each  input  class  resulting  in  a single 
output  (Heilerman);  (2)  the  module  vector  measures  (Me Tap,  Chapin), 
applied  to  deciders  and  loops  exceeding  heurlstlcally  assigned  reference 
values;  (3)  the  classification  measures  (Sullivan);  (4)  the  topological  mea- 
sures (Chen,  McCabe,  Myers);  (5)  the  measures  based  upon  length  (Halstead, 
Shooman,  Laenmel) . 

In  the  second  part  a new  measure  based  upon  information  theory  is  intro- 
duced. This  measure  is  based  on  those  aspects  of  programing  which  present 
difficulty  to  the  program.  It  is  assumed  that  the  less  frequently  an  item 
is  used,  the  more  difficult  it  is  for  the  programmer  to  use  the  item 
correctly. 

Let  pt  represent  the  probability  of  usage  of  the  1th  type  in  a program- 
ming language,  and  f ^ be  its  frequency  of  usage  in  a module  or  program.  Then 
we  can  define  a complexity  measure  for  that  module  as 

C=  -£f.log2Pi 

The  p^'s  are  the  long  term  probabilities  and  can  be  measured  over  a 

period  of  weeks,  months,  or  even  years,  and  can  be  constantly  or  inter- 
mittently updated  using  automated  techniques.  The  measure  c itself  can  be 
automatically  obtained. 

If  Cj  is  the  complexity  of  the  module,  then  we  can  define  the 

program  complexity  as  C = £ C . . 

3 J 

A number  of  different  schemes  can  be  use'  to  calculate  this  measure. 

For  instance,  all  operations  and  operands  could  be  counted.  Operations, 
such  as  CALLS,  which  Include  names  unique  to  a program  can  be  ordered  accord- 
ing to  their  frequency  of  usage.  The  module  called  with  highest  frequency 
would  be  paired  with  the  highest  probability.  A similar  technique  would  be 
used  with  operands. 

Another  technique  might  be  to  group  certain  operators  together  and 
combine  their  probabilities.  As  an  illustration  of  such  grouping,  there 
does  not  seem  to  be  any  reason  to  assume  that  addition  is  either  more  or 
less  complex  than  subtraction.  Thus  addition  and  subtraction  may  be  treated 
as  one  operator  as  far  aa  the  complexity  measure  is  concerned. 

We  may  wish  to  distinguish  among  levels  of  nestings.  A DO  statament  at 
the  second  level  nay  be  more  complex  than  one  at  the  first  level.  The 
higher  nestings  could  be  assigned  separata  frequencies  and  have  individual 
probabilltlea. 
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There  are  a number  of  advantageous  features  In  this  measure.  Since  the 
measure  Is  based  on  the  probability  of  usage  of  each  type,  the  measure  is 
sensitive  to  the  natural  tendency  of  humans  to  forget  what  they  do  not  use. 
The  measure  can  also  be  made  sensitive  to  the  different  levels  of  nestings. 
No  other  measure  exhibits  such  sensitivities.  Furthermore,  once  the  neces- 
sary programs  are  in  effect  at  a particular  installation,  the  calculation  of 
the  frequencies,  probabilities  and  the  ensuing  measure  can  be  completely 
automated. 


Program  complexity  and  Information  content.  The  purpose  of  a computer 
program  is  to  convey  Information  from  the  user  to  the  computer-operating 
system.  The  information  is  that  needed  to  specify  the  desired  mapping  of  In- 
put data  to  output  data.  Since  the  operating  system  and  the  computer  hard- 
ware usually  contain  many  commonly  used  functions  and  programs,  the  user  s 
program  need  not  contain  complete  details  of  the  desired  computation.  The 
amount  of  information  contained  In  a program  can  be  calculated  by  several 
methods,  and  it  might  well  be  identified  with  program  complexity. 


Ever  since  Shannon's  paper  of  1948  information  content  of  a message  has 
been  regarded  primarily  as  a statistical  quantity.  Suppose  a user,  or  a 
group  of  statistically  similar  users,  feeds  to  computer  a series  of  programs 
<r  <t  ...<r  . ..  These  programs  are  selected  from  a set  of  possible 


{it. 


, "2  •••1rM 


programs  1 ■■  j 
at  least  as  large  as 


}.  The  Information  content  of  each  program  is 
M 


1 

H = Y P(TT.)log,  -D  . ■ r 

O r * 2 Pr  (tt.  ) 


The  quantity  Hq  will  be  of  reasonable  else,  but  M is  astronomical  and  haoce 

there  semes  to  be  no  reasonable  way  to  calculate  Hq  directly.  There  might  be 

indirect  ways  to  estimate  H , and  in  any  event  it  is  useful  con cep tally. 

o 

Another  way  to  calculate  the  information  content  of  a program  is  to 
the  statistical  properties  of  parts  of  the  program  such  as  charac- 
ters, operators,  operands  etc.  Call  these  smaller  units  tokens  as  shown 
in  Fig.  11, 


Fig  11.  Illustration  of  Tokens 

and  consider  the  sequence  of  tokens  going  from  the  user  to  the  computer. 
Define  p (i  | Bjn"  ^ ) as  the  probability  of  the  1th  token  following  a 
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token*.  Define 


given  block  of  Bj^n  ^ 


where  A Is  the  number  of  type*  from  which  each  token  may  be  selected.  Since 

F 1*  mono tonic  non- Increasing  It  must  approach  a limit  (possibly  0)  as 
n 

n — oo  . Define  FM  as  this  limit,  that  Is 


F._  • Lim  F 

00  n “oo  n 


A 


Let  be  the  average  number  of  tokens  In  a program.  It  can  be  shown  that 


Ho  < 


A value  for  L F „ exceeding  H Indicates  that  the  program  contains  more 
av  o 

Information  than  actually  required.  This  additional  Information  might  be 
due  to  coMints,  mnemonic  labels  and  variable  names,  alternate  algorithms 
solving  the  same  problem,  different  ways  to  order  program  blocks,  different 
ways  to  allocate  memory  etc. 

Operators  snd  operands  often  alternate  In  a program,  either  because 

the  program  Is  written  that  way,  or  because  operator  and  operand  are  so 

defined  as  to  force  the  alternation.  In  either  case,  a good  choice  for  the 

token  In  computing  F Is  a operator  - operand  pair.  Since  A Is  a large 

n 

number,  the  actual  calculations  are  greatly  facilitated  If  the  probabilities 
follow  a simple  distribution  formula  such  as  Zlpf'a  law.  Work  is  In  pro- 
gress on  developing  the  relations  between  the  various  Information  measures 
with  the  theory  of  Zipf's  law  applied  to  operation  - operand  In  evaluating 
numerical  examples.  This  work  will  be  reported  In  a future  report. 


2.6  MODELS  FOR  THE  MANAGEMENT  OF  SOFTWARE  — M.L.  Shooman  and  A.  Kershenbaum 

Initial  work  on  two  models  for  the  management  of  software  were  under- 
taken. The  first  model  considers  the  development  time,  T,  to  be  the  sum  of 
programBilng  time  T and  the  communication  time  T , that  is. 
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For  Np  progri 
faces  Is 


•rs  Interfacing  with  each  other,  the  total  number  of  Inter- 


(V)  . ”p  ”■>  - » 


If  S Is  the  total  number  of  Instructions  to  be  developed,  and  T Is  measured 
In  months,  then  the  productivity  P measured  In  Instructions  perpmonth  is 


One  assumption  is  that  the  coasnmlcatlon  with  each  Interface  requires  a 
fraction  K of  T,  ve  obtain  for  the  communication  time  T 


Substitution  gives 


KIN  (N  - 1) 

E E 

2 


IN  (N  - 1) 


H X > . ■ ..  

V KN(Np-l) 

1 _ — E 

1 2 

If  on  the  other  hand  there  is  a fixed  coordination  time  for  each  Interface, 
defined  as 


_ _ months  coordination  time 
*2  program  Interface 


then  man-months  N T become 

P 

V • "T  + vfo  • 11 
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The  second  model  treets  the  question  of  decomposition  of  e problem  Into 
subprobleme  (solved  by  distinct  individuals)  end  the  attendant  communica- 
tion problems.  The  objective  Is  to  study  the  effect  of  the  shape  of  the 
system  on  Its  overall  cost  or  complexity.  As  an  example , the  decompositions 
of  P Into  case  1 end  case  2 are  shown  In  Fig  12.  In  esse  1 communication 
among  subproblems , (l.e.  the  individuals  solving  these  subproblems),  takes 
place  directly.  In  case  2 all  communication  takes  place  through  a central 
point  Pj. 

A quantitative  comparison  of  these  and  other  more  elaborate  decomposi- 
tions (Involving  more  subprograms)  are  being  studied.  The  results  will  be 
disseminated  in  a separate  technical  report. 

O 

(a)  The  Problem  P 


Fig.  12  The  Problem  P and  its  Decomposition 
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4.0  PERSONNEL  AMD  WORK  AREAS 


The  following  personnel  participated  In  the  research  activities  during 
this  reporting  period: 


M.L.  Shooman 
H.  Ruston 


D. 

Baggl 

A.  Laemmel 

E. 

Ber linger 

R.  Lipshits 

J. 

Kitalnlk 

G.  Popkln 

and  worked  in  the  following  areas: 

1.  Shooman  and  Ruston:  Models  for  estimation  of  time  needed  for  the  de- 
velopment of  software,  with  application  to  the  scheduling  and  control- 
ling a software  product.  Also  the  planning  of  a survey  on  the  use  of 
software  models. 

2.  Shooman  and  Popkln:  The  verification  of  a computer  program  through 
testing  (a  technical  report  on  this  work  was  published  during  the  re- 
porting period). 

3.  Ruston.  Ber linger.  Kitalnlk:  Application  of  information  theory  and 
software  physics  to  complexity  measures,  and  the  methods  for  evaluation 
of  the  measures. 

4.  Baggl  and  Shooman:  Automatic  test  drivers  and  their  implementation. 

5.  La camel:  Application  of  recursive  function  theory  to  program  complexity 
(a  technical  report  on  this  work  was  published  during  the  reporting 
period).  Application  of  statistical  methods  to  program  testing. 

5.0  DIRECTION  FOR  NEXT  PERIOD'S  WORK 

1.  Baggl  — Completion  of  a report  on  software  test  models  and  the  imple- 
mentation of  test  drivers  for  such  models.  Further  work  will  focus  on 
increasing  the  efficiency  and  usefulness  of  the  path  analysis  algorithm 
and  of  the  Type  1.5  driver  system. 

2.  Laemmel  — preparation  of  a report  on  statistical  program  testing, 
taking  into  account  errors  not  found  as  well  as  errors  caused  by  Im- 
proper correction. 

3.  Ruston  and  Ber linger  — Application  of  Information  theory  to  complexity 
measures  and  the  evaluation  of  such  measures. 

4.  Ruston  and  Kitalnlk  — Application  of  graph  theory  to  complexity  mea- 
sures. 

5.  Ruston  and  Mohan ty  — Simplification  of  the  technique  for  test  data 
selection.  The  present  technique  is  unyielding  and  very  time-consuming. 


36 


6. 


Shopman  and  Popkln  — Further  work  on  enumeration,  classification,  and 
characterization  of  tests  and  their  attributes.  An  effort  will  be  made 
to  correlate  practical  tests  now  in  use  with  the  theoretical  schemes 
which  have  been  established. 

7.  Shopman.  Ruston  and  Cormier  — Application  of  Markov  processes  to 
estimation  of  actual  time  needed  to  development  of  software  (taking 
the  returns  to  previous  stages  into  account).  Such  estimation  will  be 
useful  for  scheduling  and  controlling  a software  project. 

8.  Shopman  and  Lipshitz  — Further  work  on  generalizing  an  early  effort  on 
modular  and  automatic  programming  techniques. 


6.0  PROFESSIONAL  ACTIVITIES 

This  section  summarizes  the  professional  activities  of  the  research  per- 

sonnel  working  on  this  contract. 

6.1  PUBLISHED  AND  SUBMITTED  PAPERS  AND  REPORTS 

1.  C.L.  Hsu,  L.  Shaw,  S.G.  Tyan,  "Reliability  Applications  of  Multivariate 
Exponential  Distributions,"  POLY-EE-77-0361/EER122,  September  1,  1977 
(appeared  in  early  1978). 

2.  M.L.  Shooman  and  H.  Ruston,  "Software  Modeling  Studies-Summary  of  Tech- 
nical Progress,"  Final  Technical  Report,  RADC-TR-78-4,  Vol.  I (of  two), 
Jan.  1978,  A052615. 

3.  D.L.  Baggl  and  M.L.  Shooman,  "An  Automatic  Driver  for  Pseudo- Exhaustive 
Software  Testing,"  Proceedings  of  the  1978  IEEE  Spring  Compcon  Confer- 
ence, San  Francisco,  February  28-March  3,  1978. 

4.  D.L.  Baggi,  "Implementation  of  a Channel  Vocodder  Synthesizer  Using  a 
Fast,  Time-Multiplexed  Digital  Filter,"  Proceedings  of  the  IEEE  Inter- 
national Conference  on  Acoustics,  Speech  and  Signal  Processing,  Tulsa, 
OK,  April  19-12,  1978. 

5.  A.  Laemmel,  "Study  of  Recursive  Function  Theory  and  Its  Application  to 
Program  Complexity, " POLY-EE/EP-77-037/SMART  108,  May,  1978. 

6.  G.S.  Popkln  and  M.L.  Shooman,  "On  the  Number  of  Tests  Necessary  to  Veri- 
fy a Computer  Program,"  Poly-EE-78-047/SRS  113,  June  1976. 


6.2  TALKS  AND  SEMINARS 

1.  H.  Ruston,  Seminars  on  Structural  Design  and  Structured  Programming, 

Cherry  Hill,  NJ,  January  5,  1978,  N.Y.  City  April  26-27,  1978,  Toronto, 
Canada,  May  9,  1978. 
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2.  M.L.  Shooman,  Talk  on  Software  Reliability  System,  IBM  System  Research 
Institute,  March  7,  1978. 

3.  M.L.  Shooman,  "Summary  of  Modeling  Progress,"  RADC  Contract  Report, 

Griff iss  AFB,  NY,  March  10,  1978. 

4.  M.L.  Shooman,  5-Day  Course  on  Reliability,  Availability  and  Safety, 
Stat-A-Matrix  Institute,  Princeton  University,  April  24-28,  November  6- 
10,  1978. 

5.  M.L.  Shooman,  "Software  Engineering  Models,"  Computer  Science  Seminar, 
Queens  College,  NY,  May  19,  1978. 

6.  M.L.  Shooman,  "Software  Reliability  Models,"  Spring  Lecture  Series,  N.Y. 
Section  ACM,  May  18,  25,  1978. 

7.  M.L.  Shooman,  "Future  of  Computers,"  High  School  Science  Day,  Hofstra 
University,  Hemptstead,  NY,  June  2,  1978. 

8.  D.L.  Baggi,  "A  Test  Driver  for  Software  Testing,”  Seminar,  PINY,  .Farm- 
ingdale,  NY,  July  6,  1978. 

9.  M.L.  Shooman,  Series  of  four  lectures  and  four  discussion  sessions  on 
Software  Reliability,  Advanced  Course  on  Reliable  Computing  Systems, 
University  of  Newcastle,  England,  July  31-August  4,  1978. 

10.  E.  Ruston  and  M.L.  Shooman,  "Software  Models  — Some  Applications," 
Syracuse/RADC/IEEE  Workshop,  Minnowbrook  Workshop,  September  20,  1978. 

11.  M.L.  Shooman,  "Software  Reliability  Models,"  Martin  Marietta  Labs., 
Baltimore,  MD,  September  28,  1978. 

12.  M.L.  Shooman,  "Reliability  Analysis,"  Polytechnic  Graduate  Seminar  Series 
in  Electrical  Engineering,  October  5,  1978. 

13.  M.L.  Shooman,  "Software  Engineering  Models,"  Computer  Science  Lecture 
Series.  Lecture  Talk  and  discussion  leader  at  graduate  seminar.  Uni- 
versity of  Maryland,  October  23,  1978. 

14.  E.  Llpschitz,  "Automatic  Programming,"  Computer  Science  Seminar,  Poly- 
technic Institute  of  New  York,  November  8,  1978. 

15.  M.L.  Shooman,  "Engineering  Reliability,"  Polytechnic  Executive  Manage- 
ment Seminar,  Sky top,  PA,  November  13,  1978. 

16.  M.  Ruston,  "Software  Models,"  Computer  Science  Seminar,  Polytechnic 
Institute  of  New  York,  December  13,  1978. 
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6.3 


EDUCATIONAL  ACTIVITIES 


SYMPOSIA.  TECHNICAL  SOCIETIES.  AND 

1.  M.L.  Shooman  organized  a group  of  five  Interrelated  talks  on  Reliability 
Application  in  Mechanical  Engineering  for  the  Polytechnic  Graduate  ME 
Seminar,  Fall,  1979. 

2.  H.  Ruston  reorganized  three  core  courses  in  the  undergrsduate  computer 
science  curriculum  and  developed  a new  course  in  dsts  structures. 

3.  M.L.  Shooman  developed  new  notes  and  material  on  Software  Design  to  his 
graduate  Software  Engineering  course. 

4.  A.  Laeomel  developed  two  new  undergraduate  microprocessor  laboratory 
courses  and  a graduate  microprocessor  course. 

5.  M.L.  Shooman  serves  on  the  Executive  Committee  of  the  Computer  Society's 
Technical  Committee  on  Software  Engineering. 

6.  Various  research  members  (Shooman,  Ruston,  Laemmel,  Baggi , Popkln)  regu- 
larly serve  as  referees  for  Comp con,  Compsac,  IEEE  Proceedings  on  Soft- 
ware Engineering,  and  Networks,  conferences  and  Journals. 


6.4  BOOKS  AND  AWARDS 

1.  Henry  Ruston  published  the  textbook  "Programming  with  PI/ I,"  McGraw-Hill 
Book  Co.,  HY,  1978. 

2.  Martin  L.  Shooman  contributed  a chapter  entitled  Software  Engineering  to 
the  book  "Computing  System  Reliability"  being  edited  by  Brian  Randell 
and  scheduled  for  publication  by  Cambridge  University  Press,  1978. 

3.  Martin  L.  Shooman  received  the  1977  Annual  Reliability  Award  of  the  IEEE 
Reliability  Society  on  January  18,  1978. 

4.  Martin  L.  Shooman  "Software  Engineering:  Reliability,  Design,  Man- 
agement," accepted  for  publication  by  McGraw-Hill,  November  1978. 
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